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2Idt:liod and AjqciaratUB for using metadata from different * 
sources 

The iziventioii related to a method and to an apparatus 
5 for using metadata from different sources, especially for 
organizing of recordings, searching for content and access- 
ing specific recordings. 

10 Background 

The capacity of digital storage media for personal video re- 
cording grows continuously. Currently it is possible to 
store about 20 full-length movies on a single 100 gigabyte 
IS (OB) hard disk. In 2005^ it will likely be possible to store 
about 80 movies on a single 400 GB hard disk. 

Similarly^ the capacity of exchangable storage media is ex- 
panding- DVDS containing full-length movies are widely 

20 available and also rewritable DVDe (DVD-RAM, DVD-RW, DVD+KW) 
are on the market, having a storage capacity of 4.7.GB- Purr 
thermore^ the basic specification for a next generation of 
large capacity optical discs called "Blu-ray Disc" has re- 
cently been established, enabling the recording/ rewriting 

2S and playback of up to 27 GB of data on a single sided single 
layer disc, in order to further increase the capacity/ two 
or more layers may be used per side and these may be applied 
to both sides of the disc. Finally, several discs may be 
combined in a special magazine. 



This enormous amount of data requires new ways to organize 
the recordings, search for content and access specific re- 
cordings. One possible solution for this is to use so-called 
metadata, defined as data about data, for the recorded con- 
35 tent. Various industry groups and standard bodies have been 
developing metadata standards for different purposes and ap- 
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plications. In the context of metadata usage, the content to 
which metadata are related is often called "^essence''. 

The compression standards for AV streams MPEQ-l, MPEQ-2 and 
5 MPEQ-4 already contain some program specific information 
(PSI) r specified in the respective MPEG systems standard . 
Similarly/ the DVB standard used for the transmission of 
digital television signals specifies Service Information 
(DVB-SI) included in a DVB compliant MPBG-2 transport stream 
10 multiplex. 

More recently, the TV- Anytime Forum defined in the specifi- 
cation S-3 on Metadata, document number SP003vl,l metadata 
which allow the consumer to find, navigate and manage con- 

X5 tent from a variety of internal and external sourcee includ- 
ing, for example/ enhanced broadcast, interactive TV, Inter- 
net and local storage. Additionally, these metadata may also 
include information about user preferences, such as favorite 
actors or TV shows, which* facilitate automatic filtering and 

20 acquisition of content by agents on behalf of the consumer. 

Similarly, an MPEG standard for the effective and efficient 
access to multimedia content is cxxrrently under development 
[ISO/IEC 15938: Information Technology — Multimedia content 
25 description interface] - This MPEG-7 standard will offer 
metadata elements and their structure and relationships, 
that are defined in the form of Descriptors and Description 
Schemes to create descriptions of the multimedia content- 

30 

Invention 

The invention is based on the recognition of the following 
fact. Often several different metadata describing the same 
35 content may be available. However, these metadata may have 
different associated meanings (semantics) and ways of coding 
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(syntax), which are defined in the respective standards, 
e.g. the MPBG-7 or TV-Anytime standard. On the other hand, 
in systems planning to ernploy metadata, certain semantics 
and syntax of metadata or metadata fields will be more ap- 
propriate than others, depending on the properties and fea* 
tures the system shall be able to perform. a!herefore, it is 
desirable to allow the use of all available metadata, de- 
spite their different semanbics and syntax. 



10 Therefore, a problem to be solved by the invention is to al- 
low the use of metadata originating from different sources 
having different semantics or syntax. 



This problem is solved by the method disclosed in claim 1. 
15 An apparatus that utilizes this method is disclosed in 
claim 8* 

According to the invention original metadata are received 
Crom different sources and unified metadata are derived 
20 from said original metadata using one or more transformation 
rules I wherein the transformation rules specify how the con- 
tent or parts of the content of said original metadata shall 
be used to constitute the content or part of the content of 
said unified metadata. 

2S 

Advantageously/ the derived unified metadata are presented 
to a user and are processed in response to a user input. 

Advantageously, the processing results in selecting, delet- 
30 ing or modifying said unified metadata. 

Furthermore, it is of advantage to store the original meta- 
data, keep the stored original metadata unchanged and enable 
to recurr from the vmif ied metadata to the original meta- 
35 data . 
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For the transformation rules, it is advantageous to base 
them on formal and/or structural properties of the original 
metadata in all cases where the original metadata are them- 
©elf structured. 

s 

For uiistructured original metadata of textual nature, it is 
advantageous to base the transformation rules on a similar- 
ity measure which allows for a tolerant matching of phrases 
against a user-defined search phrase. 

10 

According to a further advantageous embodiment a part of the 
transformation rules covers the mapping or inheritance of 
the essence links from within the original metadata into all 
derived unified metadata. 

15 

Further advantageous embodiments of the invention result 
from the following description. 

20 Drawing 

Exemplary embodiments of the invention are described with 
reference to the accompanying. drawings / which show ins 

25 Fig- 1 generically the conversion of original metadata re- 
cords with different syntax and semantics into uni- 
fied metadata records; 
Fig. 2. a block diagram of the processing of metadata in- 
cluding the derivation of \mified metadata from re- 

30 ceived original metadata. 

Exemplary embodiments 

35 Figure 1 shows generically the conversion of original meta- 
data records with different syntax and semantics into uni- 
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fied metadata records, in the following also called Core 
Metadata. Transformations Tlf-,^ Tj,... specify rules how the 
content or parts of the content of the original metadata re- 
cords of a predetermined class shall be used to constitute 
5 the content or part of the content of a unified metadata re- 
cord. In Figure Ir the blocks ^'ODRl"' to ^"ODRk+m" syiobolize 
different classes of original metadata records, and more 
specifically, it shall be assumed that blocks ODRl and ODRk 
both are gov^emed by one specific syntax/semantics, whereas 
10 blocks ODRk+1 and ODRk+m are governed by a second, different 
syntax/semantics. All original metadata records of class 
ODRl are converted by a set of transformations Tl into in- 
stances of a first class of unified metadata records UDRl, 
As part of this conversion, the transformation Tl may man- 
is date the presence of certain parts within the instance of 

ODRl/ so that by applying Tl, not all instances of ODRl will 
actually generate an output UDRl, Also, it must be kept in 
mind, that the conversion described by Tl may mandate that 
only certain parts of the Instance of ODRl shall be taken to 
20 constitute' certain parts of the emerging instance of UDRl. 
Similarly, the same set of transformations Tl may contain a 
rule which defines that, vinder certain conditions, soma pre- 
defined parts of instances of a second class of original 
metadata records ODRk are converted into other instances of 
25 the first kind of unified metadata records ODRl, However, 
other instances of the second class of original metadata 
ODRk, if they contain certain other parts, are converted by 
a different set of transformations Tj into Instances of a 
second class of unified metadata records TJDRj • Similarly, 
30 fiirther original metadata records of classes ODRk+1 and 

ODRk-hm having a syntax/ semantics according to a second stan- 
dard are converted by the transformations Tl and Tj into in- 
stances of the first and second unified metadata records 
UDRl and UDRj . In this way, when a sufficient number of 
35 original metadata is available and the respective transfor- 
mation rules are applied, several candidates will be gener- 
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ated for ea.cJx kind of Core Metadata. In this way, a user has 
sufficient material from which he can editorially distill 
the Core Metadata he finally wants. 

Figure 2 shows an example for the derivation of Core Meta- 
data from originally received metadata as patrt of an overall 
metadata processing chain. Although the further description 
concentrates on the processing of an MPEQ-2 transport 
stream, it can easily be generalised for use in any system 
using multiplexed bitstreams coinprislng metadata, 

When recording a broadcast DVB transport stream TS addi- 
tional service information encompassed in the multiplex be- 
sides the audio and video signals is parsed from the multi- 
plex. For this purpose the broadcast signal is analyzed for 
metadata like teletext, DVB-SI etc. by a DVB metadata parser 
MD-PAR and the detected metadata are demultiplexed and ex- 
tracted* 

The extracted metadata are transformed into XML metadata de- 
scriptors XML-DS. This XML representation for the metadata 
descriptors allows to have a generic framework for different 
kinds of metadata descriptors, in other words, the metadata 
database MD-DB can store any kind of metadata as long as 
there is a transformer that converts it from its native rep- 
resentation into the XML representation. Similarly it is 
possible to perform queries on the metadata, independent of 
their respective metadata sources. 

There are different transformers for different metadata 
sources, like e.g. a transformer XML-SI for the transforma- 
tion of DVB-SI data or a further transformer XML-TT for the 
transformation of Teletext or Closed Caption data. The 
transformers transform the metadata from their native cod- 
ing, e.g. DVB-SI binary format, into an XML representation. 
This transformation is keeping the information inside the 



z&.m^.snaaz 9:49 

PD020018-Ri-25 



NR. 459 5.11X20 



7 

metadata ^^as ±s^\ It neither adds nor removes information. 
This allows a very flexible later use of the metadata. Dur- 
ing recording the metadata descriptors are accumulated and 
stored into the metadata database MD-DB. 

5 

Similarly, metadata descriptors generated manually by the 
user such as user-annotations UA may be stored after they 
have been transformed by a transformer XML-UA into XML rep- 
resentation. Finrthermore, metadata descriptors according to 
10 other standards like HTML/ TV-Ariytime/ MPEG- 7 or SMPTB are 
transformed by respective transformers XML-HTML^ XML-TVA/ 
XML-M7 or XMIi-SMPTE and stored. These metadata descriptors 
may be received via the same or other transmission patbs 
like the Internet, 

IS 

An archive system may also store metadata descriptors origi- 
nating from other recording devices axxd/or storage media. 
However, this necessitates an e^cport and import facility MD- 
BXP-IMP for metadata descriptors, such that metadata de- 
20 scriptors can be exchanged between devices and/or media. 

In order to use the metadata descriptors stored in the meta- 
data database MD-DB the inventive conversion of tbe original 
metadata into Core Metadata CM has to be performed using 
35 specific transformation srules MD-T. Here is where it becomes 
beneficial that the metadata is stored ^'as is". No informa- 
tion has been added or removed from the complete set of 
metadata information that is available, 

30 Finally/ access may be provided to all metadata via a user 
interface UI with query and browsing techniques Q for find- 
ing the corresponding video/audio stream or metadata the se- 
* lected metadata descriptor is pointing to. 

35 One example of the conversion of the original metadata into 
Core Metadata is the conversion of different metadata items 
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into title information for a given recording. Often tJie ti- 
tle information i^ the most useful and versatile single 
metadata item when it comes to archiving, searching and 
sorting of any kind of AV material. For all material, be it 
s self recorded (e.g. with a camcorder), or recorded from 
broadcast, or be it on canned media, mostly everybody will 
be able to associate a Title with it, under which the mate- 
rial can, on one hand, be recognized for later search, but, 
on the other hiand, can also be sufficiently differentiated, 
10 if necessary, against other material which might be closely 
similar. 



Corresponding to this very intuitive notion of Title/ many 
existing metadata systems comprise items which constitute a 
X5 container for Titles, although they may come under different 
names and have distinct syntaxes. 

- DVBr in its ^^DVB-Sl'' system of Service Information^ de- 
fines a short event descriptor" which *'provlde5 the naaie 
of the event and a short description of the event In text 

20 form"^. The syntax of the short event desoriptor foresees 
two variable length text fields s a dedicated one for the 
^^0vent_name^ , and another one to ''specify the text de- 
scription of the event 

- MPEG-7, in its ''Creation jDe^criptioii Scheme"^, has the no- 
25 tion of "'Title''/ as "the name of an audio-visual program'^ 

- TVAnytime, in its work on Electronic Program Guides, men- 
tions the concepts of ^"Title" as ''Textual title of the 
proffram (...) Multiple title descriptorB may he included'' 
Similar concepts of ''Episode Title'' and "Series TitJe'^ ex 

30 ist. 

- Any container for any kind of User Annotations might al- 
ways happen to be used for a user-specific Title of a 
piece of AV material, 

35 • Correspondingly, a Core Metadata scheme according to the 
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present invention, will have one olaes of Core Metadata 
named '"Title", and all the above mentioned given xnetdata 
should be used as input to derive "Title" candidates, be- 
cause - depending on what they contain - they might contain • 
B good material to be used for the above mentioned intuitive 
notion of '^ritle". Hence, the Core Metadata scheme will de- 
fine transformation rules for each of the metadata above, 
specifying in each case which part of the content should be 
inserted as the content of the derived Core Metadata item. 
10 The Core metadata schema may then provide tools such that 
the user can inspect all the ^Title" candidates, select one 
of them and/or edit their content as to match his/her per- 
sonal preference. 
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An even more illustrate example for the derivation of Ti- 
tle'^ candidates could be as follows. BPS data are transmit- 
ted according to the TV-Anytime standard and are received by 
a TV receiver or sat tpp box. A user invoices a display of 
the EPa on a television screen and selects a television pro- 
gram in the displayed BPG based on the displayed title. The 
title information aecoi:ding. to the TV-Anytime standard is 
separated and stored as a first candidate of the Core MD 
^title". After tuning the TV receiver to the selected chan- 
nel, the Sl-information within the DVB signal is analyzed 
and a second title information is separated and stored as a 
second candidate. Furthermore, the teletext signal received 
on the received channel is checked for information about the 
current program, which is stored as third title candidate. 
Since the stored title information is based on different 
standards, different transformations are applied to the ti- 
tle informations in order to convert them all into the same 
format. All converted title informations are stored, and at 
any convenient later time, they can be displayed to the user 
and the user then can select the "best" title information. 
For exan^le, for a long title two of the three title infor- 
mations may happen to be abbreviated versions while only the 
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third one may happen to be complete. The user may in this 
case select the complete title information for futwe ove; 
views of hie recordings. 



5 Metadata being defined as "Mata about data", a constituent 
part of any original metadata will always be an essence 
link, by which the metadata points to an AV essence. Prom 
this it is obvious, that parts of the transformation rules 
MD-T will advantageously describe the details how the es- 

10 sence links found on an input metadata shall be duplicated 
in order to appear on the derived Core Metadata, too. In ad- 
dition to essence links, an original metadata may also con- 
tain '"origin links" specifying where the original metadata 
was received or extracted from. Assuming that Core Metadata^ 

IS too, may have such an ^"origin link", another part of the 

transformation rule JMD-T may describe how this Core Metadata 
origin link shall be set, one possible solution being that 
this link points to the original metadata from which the 
core metadata was generated. 

20 

For original metadata which are themself structured, like 
those according to the t>VS-SI or TV-Anytime standard, the 
transformation rules MD-T will typically select only certain 
kinds of metadata for further processing, will test for the 

25 presence of certain optional structural parts of the input 
metadata, will potentially check some parts whether they 
have prescribed values, and, conditional on these precondi- 
tions being met, the transformation will extract a certain 
subset of the input metadata and embed it in a certain posi- 

30 tion of the output Core Metadata to be generated • 

For other input metadata, especially those which are un- 
structured except being indicated as textual", a transfor- 
mation rule MD-T may typically consist in evaluating a 
35 phrase similarity measure between the input metadata and 
some user-provided search phrase. Such a phrase similarity 
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measure should advantageously be made tolerant against 
slight variations of grairanar or wording, which can be 
achieved by evaluating word order and word distance, and by 
additionally converting all words to be compared into word 
5 stems using algorithms known in the art as ""stemming^'. 



The invention is especially useful for personal video re- 
cording, e.g. for the search and access of specific re- 
cordings on high- volume storage media. However i the inven- 
10 tion is also applicable to all other kinds of electronic 

multimedia content referencing and content location. The re- 
corded content may be movies and television programs but 
also every other form of audiovisual information/ such as 
arbitrary combinations of still pictures, graphics, 3D mod- 
is els, audio, speech and video. 



EfflPfangszeit 26.M8rz 9:54 



9^50 

PD020018-Ri-25 



NR,459 



S. 16/^0 




Claims 



10 



15 



20 



25 



30 



1. Method for using metadata from different sources, said 
metadata deecriblng and being linked to multimedia es- 
sence and being used for accessing said multimedia es- 
sence, cliaractierized by 

- receiving original metadata from different sources; 

- deriving from said original metadata unified metadata 
using one or more transformation rules, said transfor- 
mation rules specifying how the content or parts of the 
content of said original metadata shall be used to con- 
stitute the content or part of the content of said uni- 
fied metadata. 

2. Method according to claim 1, further characterized by 

- presenting said derived unified metadata to a user; 

- processing said derived unified metadata in response to 
a user input. 

3. Method according to claim 2, wherein said processing re- 
sults in selecting, deleting or modifying said imified 
metadata. 

4. Method according to any of claims 1 to 3, further 
characterized by 

- storing the original metadata; 

- keeping the stored original metadata unchanged? and 

- enabling to recurr from the unified metadata to the 
original metadata, 

5. Method according to any of claims 1 to 4, wherein said 
original metadata contain links, and where a part of said 
transformation rules covers the mapping or inheritance of 
said links from within said original metadata into all 
derived said unified metadata. 
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6. Method according to any of claims 1 to 5/ wherein said 
original matadata are at least partly structured and 
where said transf onnation rules are at least partly based 

5 on examining and evaluating the structure of said origi- 

nal metadata. 

7, Method according to any of claims 1 to 5/ wherein said 
original matadata at least partly consist o£ sequences of 

10 words not othesciAfise structured, where said transformation 

at least partly involves a search phrase^ where said 
transformation rules are at least partly based on evalu-* 
ating a phrase similarity measure / where said phrase 
similarity measure is based on word order and/or word 

IS distance measures, and where, prior to applying said 

phrase similarity measure, the words of said original 
metadata and of said search phrase are optionally normal- 
ized by applying a stemming algorithm. . , 

20 8. Apparatus for using metadata from different sources/ said 
metadata describing multimedia essence and being used for 
accessing said multimedia essence, characterized in 
- means for receiving original metadata from different 
sources ; 

25 - means for deriving from said original metadata unified 

metadata using one or more transformation rules, said 
transformation rules specifying how the content or 
parts of the content of said original metadata shall be 
used to constitute the content or part of the content 

30 of said unified metadata. 



9- Apparatus according to claim 8/ further characterized by. 
- means for presenting said derived unified metadata to a 
user; 

35 - means for processing said derived unified metadata in 

response to a user input. 
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Abstract 



Metadata from different sources are used to organize re- 
cordings, search for content and access specific recordings. 

5 Xnxe to different metadata sources using different standards, 
the metadata have different semantics and syntax. According 
to the invention, from the original metadata received from 
the different sources unified metadata are derived using one 
or more transformation rules- The transformation rules spec- 

.0 ify how the content or parts of the content of said original 
metadata shall be used to constitute the content or part of 
the content of said unified metadata. The derived unified 
metadata are presented to a user and processed in response 
to a user input. 

LS 

Fig. 1 
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